Method and apparatus for alignment, comparison &amp; identification of characteristic tool marks, including ballistic signatures

ABSTRACT

Systematic use of infrared imaging characterizes marks made on items and identifies the particular marking tool with better accuracy than use of visual imaging. Infrared imaging performed in total darkness eliminates shadows, glint, and other lighting variations and artifacts associated with visible imaging. Although normally used to obtain temperature measurements, details in IR imagery result from emissivity variations as well as thermal variations. Disturbing an item&#39;s surface texture creates an emissivity difference producing local changes in the infrared image. Identification is most accurate when IR images of unknown marks are compared to IR images of marks made by known tools. However, infrared analysis offers improvements even when only visual reference images are available. Comparing simultaneous infrared and visual images of an unknown item, such as bullet or shell casing, can detect illumination-induced artifacts in the visual image prior to searching the visual database, thereby reducing potential erroneous matches.

This application is a continuation of, and claims priority from,co-pending U.S. patent application Ser. No. 09/329,217 of the sametitle, which in turn claims priority from U.S. Provisional PatentApplication No. 60/087,512 filed Jun. 10, 1998.

SUBJECT OF THE PATENT

The method and apparatus of this patent involve tool markidentification, which is a type of forensic analysis aimed at comparingimages of marks made on an item with images of marks stored in adatabase to find any which appear similar enough that they could havebeen made by the same tool. The principal purpose is to find linkagebetween items associated with crimes, to assist in solving the crimes.The primary application is for analysis of ballistic items, whichincludes guns, bullets, and shell casings.

BACKGROUND OF THE INVENTION

Guns, including pistols, revolvers, semiautomatics, rifles, and shotgunsare used in the commission of thousands of crimes per day throughout thecountry. There is growing interest in tracing the source and movement ofguns. Often crimes are solved by being able to associate one crime withanother, or a crime with a particular gun, or a gun with a particularperson. Tools used during commission of a crime often leavecharacteristic marks at the crime scene, which marks can be useful inlinking together various crimes, or in linking a crime to a weapon, orin linking a crime to a criminal. The evidence that allows such links tobe developed involves particular tool marks, which each weapon makes onthe bullets and shell casings fired from it.

Because the evidence must be protected against contamination and thechain of custody must be preserved, and since it is often desirable tocompare markings obtained at one crime scene to those obtained atothers, it has become common to create images of the markings and usecomputer systems for automated comparison of the images from newlyobtained specimens with databases of previously obtained images. Onlyafter a possible match has been determined are the physical specimensviewed side by side.

Ammunition for handguns, rifles and shotguns all contain an outer casingor shell, a primer device to ignite, a fast burning material which is aform of nitrocellulose, and a projectile to fly out the barrel.Ammunition that travels through a particular gun receives scratches fromthe barrel of the gun, indentation from the firing pin, and perhapsother marks from the breech face and ejector. These tool marks arecharacteristic of the particular weapon used, and may be used to link abullet or casing with the weapon from which they were fired.

The shell casing receives marking from the firing pin hitting theprimer, from the back pressure of the gas expansion forcing the casingagainst the breech face of the firing pin housing which may have marksor defects which transfer onto the primer and/or casing. These marks maybe a result of manufacturing defects, or hand finishing done in highquality weapons. Breech face marks can be compared just as can firingpin marks, either by firing a test round or by examining the weapon whenthe weapon is available, or by comparing corresponding marks on twobullets or casings which are suspected of coming from the same weapon.If the same weapon is used to fire the same type ammunition at the sametype target from the same distance, comparable patterns will be producedon the bullet and casing. If the ammunition is changed, the patternswill be somewhat different.

Tool marks can be also transferred to the casing by the extractor andejector of semiautomatics and automatics. Such marks includeindentations and striations. The ejector is likely to gouge the casing.The extractor pulls the casing from the breech, imposing perhapsadditional striations and indents before it hits the ejector.

Class characteristic marks vary with calibre, load, material used forthe bullet or shot, bullet weight, its impact behavior, material usedfor the casing, and identification stamped into the bullet and casing.Intentional marks on ammunition include information stamped on the faceof the bullet casing during molding of the shells. Some casings andbullets also have an indented ring or rings around the circumferencecalled a canellure. These are smeared with grease or wax as sealer,making the bullet water-resistant and providing some lubrication as itis forced through the barrel. The canellures on the casing are imprintedwithin a quarter inch from the top after the bullet is inserted. Thiscrimp acts to seal the round and hold the bullet in the casing.Canellures may contain imprint information unique to the manufacturerand perhaps to a particular crimping tool.

Ammunition for handguns, rifles and shotguns all contain an outer casingor shell, a primer device to ignite, a fast burning material which is aform of nitrocellulose, and a projectile to fly out the barrel.Ammunition that travels through a particular gun receives scratches fromthe barrel of the gun, indentation from the firing pin, and perhapsother marks from the breech face and ejector, in addition tofingerprints. The hand of the person firing the weapon receives a sprayof gun residue which may be characteristic of a particular type ofweapon and ammunition. The target receives some degree of blowback fromthe weapon and the target itself, plus perhaps fragments of anymaterials through which the bullet passed.

Markings on ballistic items which are of interest to forensics include:

Fingerprints on the weapon, projectile, or casing.

Other debris and material on the weapon, projectile, or casing.

Characteristics on the projectile or casing associated with theparticular brand and type of ammunition used.

Characteristics on the projectile or casing associated with theparticular weapon used. These are called tool marks.

During manufacture, grooves are cut into the hard steel of pistol andrifle barrels, spiraling from the chamber to the muzzle. They cause thebullet to spin, which results in the bullet having a cleaner trajectoryand the weapon having more accurate targeting. The raised areas betweengrooves are called lands. Bullets are intended to be fired in aparticular calibre weapon. Nitrocellulose burns to produce an expandinggas, which drives the bullet through the barrel. The resulting heatcauses the bullet to expand and softens its surface. Lead bullets areparticularly prone to softening. The bullet is blown out of the shellcasing and forced into the barrel, which is tighter. As a result, thelands are cut into the moving bullet and the surface is squeezed intothe grooves. The high points of the barrel cause scratches in thebullet, which are referred to as striation evidence. The material usedin the bullet determines the depth of the striations.

The number, width and depth of the grooves and the angle and direction(right or left) of their spiral are determined by the manufacturer.Lands and grooves together are called rifling. Rifling marks aretransferred to the bullet as it is forced through the barrel. Some marksindicate the class of weapon used, while others indicate the particulargun or barrel used in the case of weapons with interchangeable barrels.

Most weapons other than Colt use a right twist. The number of lands orgrooves typically is four to seven. The width varies depending on thenumber and the calibre. The degree of twist is measured by the distancetraveled during one complete rotation of the bullet. A typical Colthandgun can be described as a left twist, six lands, 1/12. Most handgunbarrels are shorter than 12 inches. Therefore, the energy transferred tospinning the bullet is not as great as in a rifle.

When the ammunition used is smaller than that designed to be used in aweapon, the result is loss of energy and penetration. In addition, sincethe bullet is loose within the barrel, inconsistent striations willoccur on various bullets fired through the weapon.

Maintenance done on a weapon can alter the characteristics imposed onits ammunition. Use, cleaning, corrosion, and intentional damage to abarrel can all affect the ability to match ammunition used in it overtime. Rust or corrosion will alter fine details. Some semiautomatics andautomatics have interchangeable barrels. After exiting the barrel, thebullet may receive additional distortions as it passes through variousmaterials. The result may be to destroy the forensic value of thebullet, or may be just to require further analysis.

The shell casing receives marking from the firing pin hitting theprimer, from the back pressure of the gas expansion slamming the casingagainst the breech face of the firing pin housing which may have marksor defects which transfer onto the primer and/or casing. These marks maybe a result of manufacturing defects, or result from the hand finishingdone in high quality weapons. Breech face marks can be compared just ascan firing pin marks, either by firing a test round or by examining theweapon when the weapon is available, or by comparing corresponding markson two bullets or casings which are suspected of coming from the sameweapon. If the same weapon is used to fire the same type ammunition atthe same type target from the same distance, comparable patterns will beproduced on the bullet and casing. If the ammunition is changed, thepatterns will be different.

Shell casings are ejected immediately from automatic and semis, and sofall close to where the firing occurs. Revolvers retain the casingsuntil intentionally ejected, and so are often carried away from thecrime scene. Tool marks can be transferred to the casing by theextractor and ejector of semis and automatics. Such marks includeindentations and striations. The ejector is likely to gouge the casing.The extractor pulls the casing from the breech, imposing perhapsadditional striations and indents before it hits the ejector.

Class characteristic marks vary with calibre, load, material used forthe bullet or shot, bullet weight, its impact behavior, material usedfor the casing, and identification stamped into the bullet and casing.Intentional marks on ammunition include information stamped on the faceof the bullet casing during molding of the shells. Some casings andbullets, particularly lead, also have an indented ring or multiple ringsaround the circumference called a canellure. These are smeared withgrease or wax when first inserted into the casing. The material acts assealer, making the bullet water-resistant and providing some lubricationas it is forced through the barrel. The canellures on the casing areimprinted within a quarter inch from the top after the bullet isinserted. This crimp acts to further seal the round and hold the bulletin the casing. Canellures may contain imprint information unique to themanufacturer and perhaps even to a particular crimping tool.

Types of ammunition are distinguished by their size. All handguns aredesignated by calibre in either inches or millimeters. For example a0.22 calibre is 22/100inch. The length of the shell varies to increasethe capacity of gunpowder. A 0.22 may be short, long or long rifle, plusa magnum load. All magnum loads contain more gunpowder and may propel aheavier bullet. Magnum weapons are always designed to be heavier tocontain the increased force. A magnum weapon can chamber a regular load,but a regular weapon cannot chamber a magnum round.

Some 0.22 handguns and rifles can chamber only the short, some canchamber short and long, and others can handle all three. The 0.22 shortand long have the same weight bullet, while the long has more powder.The long rifle has more powder still and a heavier bullet but still isnot considered a magnum load. All 0.22 rounds of these types are rimfire primers, rather than center core primers. Some high-powered riflescan proper 0.22 calibre bullets with center core primers, but the shellcasings are huge relative to this group.

The 0.25 calibre pistol purse gun is designed to be easily concealed.There are 0.32 calibre revolvers and semiautomatics. The 0.32 calibrerim fire primers were used in early revolvers, but all modern rounds arecenter core primers.

There are two types of ammunition for 0.38 calibre (0.357) revolvers;the 0.38 S&W and the 0.38 Special. The 0.38 Smith and Wesson is ashorter shell with smaller load. It has been superceded by the morepowerful 0.38 Special, designed for law enforcement use. The maximumstopping power for the 0.38 calibre became available when the 0.357magnum was marketed. The weapon fires a heavier bullet, with longercasing that contains additional gunpowder. However, when firing a solidlead bullet, the weapon proved a hazard to unintended secondary targets,because the penetrating power was too great.

For police use, a combination of stopping power and safety are desired.Several 0.38 calibre bullets are designed to impact only the firsttarget, transferring all the kinetic energy to it. Hollow point bulletswere designed for this purpose. A hollow point is a soft lead bulletencased in a thin steel jacket. The point is actually an opening withthin lead walls. On impact the wall flare out. The final shape of thebullet, in the side view is similar to a mushroom. Hence the term“mushrooming” is applied. Hollow points don't necessarily mushroom everytime, and can still harm secondary targets, such as by passing throughsoft tissue of the target and injuring someone behind.

Glasser bullets were developed with a prefragmented round constructed ofa thin metal skin packed with very small lead particles suspended inliquid Teflon. The bullet's weight is equal to that of a solid leadbullet, resulting in equivalent kinetic energy. When the Glasser hitsany resisting surface, the thin walls flatten out, distributing the tinylead particles over a wide area. A bullet hitting a human or animalpenetrates and opens up, transferring all its energy to the inside ofthe target, having an elephant gun effect. The Glasser offered improvedsafety since all energy was expended when the bullet hit something.Errant shots striking pavement would not ricochet. Bullets hitting ahouse would not penetrate walls. Other manufacturers also offerprefragmented bullets.

Shotgun shells are different from handgun and rifle ammunition. Althoughsolid metal casings have been used, they are now mostly replaced by acombination casing with a metal face and base connected to a plastic orwaxed cardboard cylinder. Shotgun shells are available in gauges 10, 12,14, 16, 20, and 410. The cylinder is crimped shut, sealing in theprojectiles, shot and other materials with the gunpowder. The facecontains manufacturer's information and markings and houses a centercore primer. The actual projectiles vary. The shot, which wastraditionally lead, is gradually being replaced by steel balls. Leadshot has been cited by environmentalists as causing lead contaminationin waterways.

Current Technology for Tool Mark Identification

Firearms examiners can study the shell casings collected at one crimescene and determine the number of different weapons involved. They cancompare the casing to those from other sources. They can also comparebullets to determine the possibility they came from the same weapon, andthey can analyze weapons for characteristics which might match bulletsand casing fired from it. Current examinations of firearms, bullets andcasings use low-powered stereo binocular microscopes combined withhigh-energy illumination. The systems incorporate computers to assist infinding likely matches between a new casing or bullet and databases ofones previously collected, selecting potential matching items. Althoughsomewhat automated, current techniques are still labor intensive, actingonly to select likely matching items for manual review by a ballisticsexpert. The FBI and the ATF have each established a network ofcomputerized systems to support the identification of guns, bullets, andcasings; allowing member labs to share databases.

The Drug Fire program started by the FBI in the 1990's has established anetwork of computers in more than 40 forensic labs that exchange breechface striation information. Each member agency is responsible forclassifying all its cases, and placing the information in computer. Anymember agency can then compare evidence from its case to all other casesin the system. A second system, Bulletproof developed for the ATF,records striations on the bullet while the casings are categorized by asystem called Brass Catcher. The national scope of these two programs isintended to assist in investigations of drug-related gangs with nationalnetworks.

Baldur (U.S. Pat. No. 5,390,108) presents a computer-automated bulletanalysis apparatus. A microscope obtains and amplifies optical signalsrepresentative of the characteristics of the surfaces, and the opticalsignals are converted first to electrical signals and then to codeddigital representations. The coded representations are stored in memoryand are matched against one another to see if there is a match betweenthe bullets. The ATF system is based upon that apparatus.

Land marks of 45 calibre bullets are normally photographed at 10×magnification and then enlarged to 4×, with at total enlarged ratio of40×. 22 calibre long rifle bullets are commonly photographed at 20×,resulting in a total enlarged ratio of 80×. The reproducibility of landmarks is better than for groove marks, especially for jacketed bullets.The width of groove marks is very wide compared to land marks in somecases; such as with 45 calibre bullets fired from an M1911A1semi-automatic pistol. Imaging multiple groove marks at one time isdifficult due to the need to focus at different depths.

Examiners compare the contour of the bullet surfaces using a comparisonmicroscope. Not all of the bore surface characteristics will bereproducibly transferred even on consecutively fired bullets. Inaddition, there are always extraneous markings, which are not due to thegun bore surface, which an examiner must ignore.

Achieving proper illumination for detailed imaging is difficult withcurrent matching systems. The metallic nature of the items causes glareand reflection to interfere with good imaging. Use of glancing light tohighlight the relief structure interferes with distinguishing lands andgrooves. Shadows create or mask features, and the depth of striationsand indents cannot be determined. The automated analysis engines arecomputationally intensive and may be confused by manufacturing marks,shadow, glint, and focus. They may not distinguish between individualcharacteristics peculiar to the weapon, and manufacturing marks orincidental marks of no significance. The imagery and therefore theanalysis is prone to variations due to differences in initial alignment,and to inconsistent lighting strength and direction from day to day andfrom lab to lab and technician to technician.

Current matching systems do not have the desired accuracy, speed, costeffectiveness, and ease of use. Due to the 3-D nature of the ballisticitems, achieving proper illumination for detailed imaging is verydifficult. The metallic nature of most of the items causes glare andreflection to interfere with good imaging. Use of glancing light tohighlight the relief structure interferes with determining which marksare lands and which are groves. Shadows create or mask features, and thedepth of striations cannot be determined. Details of the firing pinindentation cannot be seen without causing significant glare and shadow.

Current systems are not optimized for networking among many differentusers each having their own imaging system and sharing databases. Imageorientation, focus, histogram, and size are not standardized. Currentapparatus allows for variations in settings of focus, brightness,orientation, and size of the image according to the judgment of theuser. The automated analysis engine is computationally intensive. It isconfused by manufacturing marks, shadow, glint, and focus. It cannotdistinguish between individual characteristics peculiar to the weapon,and manufacturing marks or incidental marks of no significance.

The imagery and therefore the analysis is prone to confusion due toinconsistent and aging lighting strength and direction from day to dayand from lab to lab. Since there are no registration indices on thefired bullets or casings themselves, the matcher must try a range ofrotations and translations for each potential match. Adopted rules, suchas aligning the firing pin blow out to 3:00 (three o'clock) are of someassistance in manual placing of casings into the microscope holder.However, that feature is not always present in casings, and when it is,that procedure still allows for variations in rotational position of 5oor more. Manually aligning striations to the 9:00-3:00 horizontal isalso prone to individual rotational variations on the order of 5o.

The current DrugFire system does not automatically extract and matchfeatures by their degree of significance, although it provides somemanually assisted techniques for highlighting regions of interest. Thecurrent systems are designed to be a filter against the database,locating likely matching items in the database and presenting pages of25 images at a time on a large screen monitor for a ballistics examinerto review and interactively compare. His comparison involves manuallyaligning two images on a split screen and looking for matching lines.Such matches may not be obvious without extensive manipulation ofcontrast, brightness, rotation, and translation of the two images.

A significant limitation to the automatic determination of matchingimages in a large database is the problem of distinguishing lands andgrooves due to variations in the strength and directionality of theillumination. Striations may also run together, confusing the count.Other illumination-induced artifacts may also be created, particularlyin the primer area about the firing pin indentation. Imprecisemeasurement of firing pin position and lack of detail on the shape ofthe indent, end points and width of striations, and other specificfeature characteristics also result from illumination variations. As aresult, visual images do not currently provide the capability for largedatabase partitioning and searches based upon extraction andcharacterization of features.

The need is for a ballistics matching technology which is faster andcheaper than current techniques, which finds more correct matches withless manual intervention by a ballistics examiner, and which can exploitthe current databases and enhance the performance of existing ballisticsidentification systems. The goal is to increase the capacity of the lawenforcement community to identify increasing numbers of ballistic items,faster and with greater accuracy, and at minimum cost both in terms ofmanpower and system expenses.

The situation is analogous to that of fingerprint matching. Asincreasingly large databases are created, techniques are needed topartition those databases and to perform matches based upon featurecharacteristics, with full image comparison being performed on only asmall percentage of the database considered candidate matches. Due tothe nature of ballistic items and the distortions to which they aresubjected, and to variations in illumination resulting from aging,positioning, focus, personnel, and laboratory differences, visualimaging does not provide sufficient consistency and reproducibility offeature location and characteristics to facilitate such precisematching.

DEFINITIONS

Gun—device with barrel which fires a projectile, including pistols,revolvers, semiautomatics, rifles, and shotguns.

Bore—diameter and inside surface of a gun; it may be smooth or grooved

Riflings—spiraled, grooved markings inside a barrel

Trigger—mechanism used to fire the gun by releasing the hammer whichstrikes a firing pin housed inside the breech block

Firing Pin—mechanism driven into the primer area of a cartridge toinitiate firing the projectile,

Primer—firing pin strike can be center fire or rim fire

Cartridge—case containing the projectile or bullet; it may be reloadedfor repeated use

Forensic Item—any item used as a weapon, or item associated with a crimewhich may display marks related to the crime, including marks made on ahuman body

Ballistic Item—gun, bullet, shell casing, or other weapon, projectile,or element including the hands which may cause or receive marks orresidue associated with firing a weapon

Infrared camera—an imaging device whose detectors are sensitive in aspectral band between 1 and 14 micron; the infrared camera may have anassociated spectral filter which blocks or transmits segments of itsband.

Active infrared imaging—heating or cooling the subject of the infraredimage, including by sunlight, hot or cold air, immersion in fluids,application of flame or ice, or other means.

Image sequence—one or more images of a forensic item in which the camerais focused on specific features of the item

Reduced image sequence—an image sequence formed by selection of imageswhich are of keenest interest to a ballistics examiner

Features—extended marks on a forensic item visible in visual or infraredimagery caused during the manufacturing process or during use. Theyinclude pits, scratches, embossing, gouges, residue, and machining flaws

Extracted feature sequence—a processed image sequence or reduced imagesequence in which only features are shown

Image sequence focus montage—also referred to as a montage—combinedsharp focus portions of multiple images in a sequence to make one ormore composite images

Class characteristics—features related to a particular type of weapon orammunition and include: calibre of weapon, type of ammunition, type ofcasing, number of lands and grooves and twist angle.

Tool Marks—residual markings left on an item which has been manufacturedor worked using a tool. In the case of a ballistic item, it can be leftby the manufacturing process, by incidental use, or by use as a weapon.

Weapon characteristics—features related to a particular weapon and mayinclude firing pin impression, breech face marks, ejector gouge,extractor mark, striations, land and groove locations

Residue—materials deposited on a ballistic item, including blood, oil,gun powder, dirt, grease, fingerprints, body tissue, and buildingmaterials.

Manufacturers Mark—embossed or indented markings made during themanufacturing process, on cartridges this would commonly include typeand calibre of ammunition such as “30-30 WIN”

Orientation—position of the ballistic item relative to the axes of thecamera

Siblings—two bullets, bullet fragments, or shell casings known to havebeen shot from the same weapon with the same barrel.

INTRODUCTION TO THE INVENTION

Infrared imaging of guns, fired bullets and spent casings can providedetailed definition of characteristic features free from the shadowing,glint, and focus blur associated with visible light imaging. Although IRimaging is normally associated with measurements of temperature, theimagery from an IR camera results from emissivity and geometryvariations in addition to thermal changes. The tool marks which are offorensic interest on bullets and casings are associated with significantemissivity differences. In particular, deep grooves such as firing pinindentations and primer edges create highly detailed observable featuresin IR imagery even at ambient temperature. Use of IR imaging forballistics identification is therefore not intended to measuretemperature differences, but rather to display and analyze emissivityand geometric differences associated with indentations and embossedfeatures of the ballistic item.

The features seen in the infrared images do not vary in position orextent as a function of illumination as do visible images. Thereforethey provide stable references for alignment with other images forcomparison purposes. Additional details can be obtained by active IRimaging, in which the temperature of the gun, projectile or casing isvaried. This provides a sequence of images in which certain features ofinterest are particularly apparent at specific temperatures. Thisprocess is suggested only when it is necessary to enhance very faintfeatures.

Summary of Benefits of IR Ballistics Identification

Infrared Imaging in Ballistics Identification

The class characteristics of the ammunition and weapon include: calibreof ammunition, type of weapon, number of land marks, direction of twistof land marks, land mark width, angle of twist of land mark. All of thisinformation can be obtained from infrared images, given that sufficientimages at the correct temperature ranges are available.

The individual characteristics of the ammunition and weapon include theparticular surface condition and shape of the bore, the particularstriations imposed on the bullet, and the particular markings imposed onthe casing. These characteristics can also be determined using infraredimages. Even when caseless Teflon bullets are used, active infraredimaging may exploit minute markings transferred to the bullet by thebarrel and firing charge. A rotating IR probe, similar to endoscopiccameras in medical use, can produce a detailed image of a gun barrel,which can then be matched against casings and bullets which aresuspected of having been fired from that gun. In some instances, thismay be preferable to making test firings from the weapon; particularlyif the weapon is considered unsafe, or if it is necessary to preservethe weapon in its present condition as to residue etc.

The weapon itself should display identifying information such as themanufacturer's name and logo and serial number. If the serial number hasbeen removed, use of infrared imaging can assist in recovering theinformation, since the emissivity of the area will be affected byattempts to file off or chemically etch off the serial number. Byheating and cooling the weapon, a number of corresponding but differentimages can be obtained and all used to assist the reconstruction effort.

Characteristic weapons-identifying markings to be imaged and compared,include:

Scratches made on the bullet by the lands and grooves

Scratches made on the shell casing by the extractor and ejector ofsemiautomatic weapons

Indentation on the casing made by firing pin and breech face.

Indications of refilling and reusing the casing

Lands and grooves within the gun barrel

Trace items in residue deposited on the shooter's hand, within the gun,on the victim, on the bullet and casing, and on other targets.

Heat conduction analyses can be performed by heating the ballistic itemand imaging it while it cools. Estimates of the depth and volume ofindentations, striations, and gouges can be made based on their coolingrates. The material composition of each area of the item must beconsidered.

Evaluation of Ballistic Matching Systems

Current systems, such as DrugFire, compare an unknown ballistic item'simage to all corresponding images in a database, producing a correlationvalue for each. Database images are then re-ordered based upon thatcorrelation value, with the highest correlation ranked first. Incontrolled testing where the identities of all siblings are known, ameasure of the accuracy and efficiency of the matching engine is theposition of siblings in the re-ordering. The results shown below in theleft column are taken from the DrugFire system manufactured by MnemonicsSystems Inc. The right column shows the use of a different matchingengine (FlashCorrelation® patented by the inventor) with the same visualimage database as used by Mnemonics. 1157 shell casings from 229 weaponswere used for the tests. The images were all taken with a conventionalvideomicroscope camera. MIKOS did not have the opportunity to collectits own images of the casings. Therefore, no infrared images wereobtained or used for this comparison test. The purpose of this table ismerely to show how ballistic matching systems are evaluated. In asmaller test, the use of infrared imagery produced significantadditional improvement in position of siblings over the use of visualimagery, with nearly all siblings clustered at the very top of theranking.

“P” is a ranking order from 2% to 100%. Each image in the database iscompared against every other, and the database is reordered by thedegree of similarity with the target image. Each sibling of the targetimage (images of other casings fired from the same weapon) is located onthe reordered list and included in the corresponding P value. Ideallyall siblings would occur at the top of the rankings. The process isperformed for each image in the test set database in order to obtain astatistical assessment of overall system performance.

Best indication of the relative performance of two systems would be tocompare their results at P=2% since as P gets larger towards 100% anymatching technique would produce similar results. In particular, atP=100% all matching techniques achieve 100% inclusion. Mnemonics did notreport rankings for P less than 10%. If we extrapolate the Mnemonicsresults to 2%, the expected value was 25%. Comparing that with the 46%MIKOS value indicates that the MIKOS matching engine achieved 80% moredetections at 2% than did Mnemonics.

The intent of the ballistics matching system is to find the best matchesin the database and display them to a ballistics examiner who makes thefinal determination as to whether a match exists. The goal is tominimize the work of the examiner without sacrificing accuracy. Thegrowing size of current databases requires a higher degree of automationto find links between crimes and suspects before either the statutes oflimitations or the suspects expire.

SUMMARY OF THE INVENTION

The present invention relates to a method and apparatus for comparingpatterns of marks found on ballistic items (including guns, bullets andshell casings) to determine whether they have common characteristicswhich would indicate that the bullet or casing has been fired from aparticular weapon. The invention provides the ability to rapidly matchimages of an unknown ballistic item with a database of images ofballistic items including guns or ammunition fired from known weapons orunknown weapons, providing linkage between weapons or crimes or otherevents from which the guns, bullets or casings were recovered.

IR Imaging Provides Consistent Feature Extraction and Characterization

Infrared imaging of guns, fired bullets and spent casings can provideconsistent and detailed definition of characteristic features free fromthe shadowing, glint, and focus blur associated with visible lightimaging. Well-defined infrared imagery from casings and bullets isobtained even with the item at ambient temperature. The apparenttemperature differences seen are due to variations in materials, andemissivity differences associated with geometry factors, and depth ofmarkings. Therefore, the firing pin indentation, its surrounding area,the primer area, and the outer ring will all appear to be at differenttemperatures, although all are at room ambient. Use of the thermalimaging IR camera is therefore not intended to measure temperaturedifferences, but rather to extract features based upon emissivity andmaterials differences. The resulting features can be automaticallycharacterized, classified, and then matched against portions of thedatabase having similarly characterized features. Since the features donot vary in position or extent due to imaging conditions, alignment ofcandidate matches is facilitated by

Active Infrared Imagery Enhances the Features but is Optional

Active IR imaging, in which the temperature of the gun, projectile orcasing is varied, provides a collection of images at the varioustemperatures. Certain features of interest are particularly apparent atspecific temperatures. Features associated with different materials suchas the primer, or with emissivity differences such as caused byabrasion, are more well defined when viewed with an IR camera. For agiven type of gun or ammunition a set of temperatures can be determinedsuch that the sequence of images resulting from heating or cooling thegun, projectile or casing to those temperatures provides an optimizeddata set for matching against a database. Matching can be done usingstandard image processing techniques such as optical or digitalcorrelation, or using proprietary techniques such as Flash Correlation(U.S. Pat. No. 5,583,950 Method and Apparatus for N-dimensional ImageIdentification and Analysis.

If the ballistic item is heated and then imaged as it cools, thetechnique provides a sequence of images at various temperatures for eachitem. At certain temperatures, depending upon the composition materialof the item, specific markings are enhanced. For example: serial numberson a gun, manufacturers markings on a shell casing, striations on abullet caused by the gun barrel, fingerprints, firing pin indentationand breech face marks on a shell casing can be characterized by size,shape, and relative location in a sequence of infrared images, as afunction of imposed temperature. A feature template can be produced foreach image in each sequence, by thresholding hot and cold values orother standard feature extraction techniques commonly used in imageprocessing.

Feature Extraction and Characterization Provides Database Partitioning

Any extended area of the infrared image with apparent temperaturedifferences from the surrounding area may be considered a feature. Usingthe sequence of images or extracted feature templates, significantfeatures can be extracted and characterized prior to use of the matchingengine. Characteristics of the infrared features such as area,dimensions, depth, focus setting, variation in depth and apparenttemperature, shape, inclusion of subfeatures, apparent temperature,location relative to center and edges of the ballistic item, ratio ofperimeter to area, distance and vector to other features, persistence orexclusion over a range of imposed temperatures, persistence or exclusionunder specific spectral filters may be computed and used to characterizethe set of features seen in an image sequence or montage. If visualimages are also being processed, then the visual characteristics offeatures seen in the infrared may also be used to characterize thefeatures for a particular ballistic item.

By storing feature characteristics in a relational database, possiblematches can be more quickly located. In general it is not necessary tofind all matching ballistic items, such as all sibling casings. A singlepositive match provides links to others already in the database. As anexample, for an unknown casing, determine the characteristics of: (1)distance from firing pin indentation to the center of the casing, (2)depth of indent, (3) angle of breech face marks relative to the linefrom firing pin indent to center, and (4) existence of ejector gouge.Then locate other items in the database which have some or all of thesame characteristics. This is referred to as partitioning the databaseinto candidate matches and other items. Candidate matches are thensubjected to further processing. The corresponding image sequences ormontages can be compared and those not sufficiently similar excludedfrom the candidates list. Remaining candidates are then presented forreview by a ballistics examiner.

Selection of features to be characterized, and the characterizationprocess, can be fully automated or manually assisted. Partitioningsignificantly reduces the search time required to look for matches, butrequires knowledge about the variations which may occur in firings of aparticular weapon. For example, changing the ammunition size or typeused will change the markings imposed by the weapon. Therefore, inconducting a search against a database, to reduce the occurrence offalse negative results, the criteria for including a database item as acandidate matching item must be considered relative to possiblevariations such as: whether the weapon has interchangeable barrels,whether it might be used with different sized ammunition, whether itmight have been cleaned, whether it might have had heavy use between thedatabase entry and the current characterization.

The precise extraction and matching of feature characteristics which isthe method of this patent cannot be effectively performed on visualimages due to the nature of the ballistic items, which produceillumination artifacts such as shadows and glint, and cause confusion oflands and grooves, and obliteration of fine details in deep markingssuch as firing pin indentations.

Class characteristics of ammunition and weapons include: calibre ofammunition, type of weapon, number of lands, direction of twist of landmarks, land mark width, angle of twist of land mark. All of thisinformation can be obtained from infrared images.

Individual characteristics of the ammunition and weapon include: theparticular surface condition and shape of the bore, the particularstriations imposed on the bullet, and the particular markings imposed onthe casing. These characteristics can also be determined using infraredimages. Even when caseless Teflon bullets are used, active infraredimaging may exploit minute markings transferred to the bullet by thebarrel and firing charge. A rotating IR probe, similar to endoscopiccameras in medical use, can produce a detailed image of a gun barrel,which can then be matched against casings and bullets which aresuspected of having been fired from that gun. In some instances, thismay be preferable to making test firings from the weapon; particularlyif the weapon is considered unsafe, or if it is necessary to preservethe weapon in its present condition as to residue etc.

The weapon itself should display identifying information such as themanufacturer's name and logo and serial number. If the serial number hasbeen removed, use of infrared imaging can assist in recovering theinformation, since the emissivity of the area will be affected byattempts to file off or chemically etch off the serial number. Byheating and cooling the weapon, a number of corresponding but differentimages can be obtained and all used to assist the reconstruction effort.

EXAMPLES OF CHARACTERIZATION OF FEATURES

In matching casing faces, the database can be characterized accordingto:

Distance off center of firing pin indent.

Depth of firing pin indent.

Percent of face area having breech marks.

Depth of breech marks.

Angle between breech marks and firing pin vector, which connects thedeepest indentation point with the closest spot on the casing edge.

Position of ejector marks relative to firing pin and breech face marks.

Images in the database can be oriented such that the flare-out of theprimer is pointing to 3:00 (three o'clock) if it is not symmetrical; ifit is symmetrical, then the firing pin indentation vector is set to bepointing to 3:00 if it is off-center; if it is also centered, then thebreech marks are set parallel to the horizontal (3:00-9:00).

For bullets, the database can be characterized according to:

Calibre of bullet

Material

Length

Number of twists

Length of striations of maximum depth

Number of striations vs. Length

Relative positions of end points of striations.

For casing sides the database can be characterized according to:

Calibre of bullet

Material

Presence of ejector mark.

Presence of extractor mark.

Number of striations vs. Length

Images in the casing sides and bullet databases should be formed ascomposites of the multiple frames taken as the bullet or casing isrotated. The composite images can be oriented so as to align striationswith the horizontal plane of the image. Due to spiraling of the landsand grooves, image segments must be composed to create an image of theresulting striation pattern from the various segments imaged.

For images derived from thermal probes of gun barrels, the database canbe characterized according to:

Calibre weapon

Twisted bore or not.

Number of lands and grooves

Angle of twist.

Relative positions of start and stop points of lands and grooves.

The resulting strip image should be formed into a 2-dimensional image asif the gun barrel were sliced open and laid flat, and should be orientedsuch that the lands and grooves are horizontal in the image.

Detection of Illumination-Induced Artifacts in Visual Images

Since existing databases of ballistic images are all taken in thevisible spectrum, utilizing those databases requires that new imagesalso be taken in the visible domain. Due to the superior advantages ofinfrared imaging, it is anticipated that at some time in the future IRballistic imaging will become standard. Meanwhile, IR images can beginto be collected for current and future use, and also to aid inexploitation of the existing databases.

By imaging in both IR and visible bands, extracting and characterizingfeatures in each band, and comparing the results, apparent visiblefeatures which are in fact illumination-induced artifacts can beeliminated from the characterization of a ballistic item prior tomatching against the database. Any true visual feature will also appearin the IR image. Eliminating artifacts will reduce the false positiveerror rate in visual matching.

True visible features such as striations may suffer from apparentreversal of deep and tall (e.g. land and groove) markings due to theangle of illumination. The comparable IR feature can be used as areference to locally vary the gray-scale allocation (look-up table orLUT) in the visible image to best match that of the IR image. The resultwill standardize the appearance of the striations in the visible image,and improve the accuracy of the matching system.

Separation of Manufacturers Marks from Weapons Marks

Certain markings imposed by a manufacturer, such as the calibre andmanufacturer's identity on shell casings, have a characteristic depth orheight which facilitates separation of those markings based upon theirapparent temperature difference in infrared images. The distinctionbetween those features and others may be further enhanced by heating orcooling the ballistic item. A template may be made from the infraredsequence which can be used to separate the manufacturers and weaponsmarks in the corresponding visible images. This processing step allowsfor separate consideration of the two sources of marks, with therealization that the two types of marks may overlap and so subtractingone type may create changes in the feature characteristics of the otherwhich must be accommodated in the matching engine.

Infrared Images to Identify Residue in Visual or Infrared Image Matching

Infrared imaging can employ optical filters to assist in determining thepresence and composition of gunshot residue, as well as blood, bodytissue, building materials and other matter deposited on bullets duringtheir trajectory. Nitrocellulose forms carbon and the oxides of nitrateand nitrite. Other components may contain barium, antimony, cooper, andlead oxides or salts. Center core primers, found in most ammunition,traditionally have contained all four chemicals, but lead is now lessprevalent, and rim fire primers, used almost exclusively for 0.22calibre, often have no barium or antimony.

By the use of filters, the presence of residue on the ballistic item canbe determined and classified. Apparent features of the ballistic itemwhich are actually residue can be ignored for purposes of automaticmatching against a database. This is particularly useful in the casewhere residue produces an apparent feature or obliterates existingfeatures. If residue is detected, the system can provide instructions tothe system user that the particular ballistic item needs to be cleanedprior to further imaging.

Infrared imaging of the hands of suspects can also utilize spectralfilters to search for gunshot residue. This provides a possibleimmediate testing capability when a firearms investigator is notavailable, or the scene is too contaminated to facilitate clean liftingof debris from the suspect for later analysis. By combining infraredimaging of the hands and face of the suspect, the issue of chain ofcustody of the hands samples is eliminated, since infrared images ofeach person's face is unique. If the infrared camera annotates date andtime, it minimizes the level of training and time required at the crimescene. Lift samples can be taken and studied via chemical analysis usinga scanning electron microscope, as is traditionally done, in addition tothe infrared analysis. For this application, an IR camera with automaticsequencing of optical filters and video or digital output recordingprovides the best setup for archival purposes and to support futureanalysis.

Application of Infrared Imaging to Advanced Weaponry

For the future, increasingly sensitive imaging and matching techniquesare required: Teflon-coated bullets do not pick up land and grooveimpressions well. Caseless bullets have been developed which leave noshell casings after firing. The weapon used is an electronic 0.223calibre rifle, called Lightening Fire, made by Jaguar Sport. The shellexists but is constructed of propellant only. When the trigger ispulled, an electric charge, not a firing pin, detonates the propellant,burning the entire casing. The combination of the self-destroying shellcase and Teflon bullet will provide very little evidence for evaluation.Active infrared imaging of such bullets may provide sufficientinformation for identification and matching.

Image Montage Creation

A composite image or images can be created by combining the best focusedlocal image of each true feature of a ballistic item to create an imageof the entire item in which each feature is in its relative position andin focus. The montaged images can be displayed to the ballisticsexaminer and/or first used in automated correlation processing todetermine candidate matching items. This step can be performed foreither visible or infrared image sequences and matching procedures.

DESCRIPTION OF THE APPARATUS

The apparatus consists of the following components:

1. Infrared imaging microscope having radiometric calibration for eachimage frame output, such that a given gray level corresponds to aparticular apparent temperature. Variable focus optics allow fordetailed in-focus imaging over the full range of depth and height of theballistic item features.

2. Shroud to avoid ambient light striking the ballistic item to causereflections or heating.

3. Holders for shell casings such that the back end is viewed normal tothe camera axis.

4. Holders for shell casings such that the side is normal to the cameraaxis and the casing can be rotated for a full 360o image.

5. Holder for essentially undeformed bullets such that the bullet can berotated and all of the surface can be imaged normal to the camera axis.

6. Holders for deformed bullets such that multiple images can be takenwhich will encompass the entire surface area. Assume the bullet may haveno obvious axis or center; the need is to completely image the surface.Multiple positionings may be needed.

7. Holder for gun which facilitates operation of an infrared probe whichhas a helical scanning rotating head and forms an infrared image of theinside of the barrel of a gun as it cools.

8. (Optional) Heater subsystem which can be on-line: radiant heat lamps,hot air, direct contact electric heat; or off-line: boiling in water orother liquid, heating in a controlled oven.

9. (Optional) Spectral filters, which can be used to obtain additionalimage information such as the composition of the ballistic item,including material of the casing, material of the bullet, type ofprimer. Alternately, the system user can determine thesecharacteristics.

10. Feature Extraction Engine, which can be image processing software,firmware or hardware to locate local areas of interest in the image.

11. Feature Characterizing Engine, which can be image processingsoftware, firmware, or hardware to perform measurements of the featuresand associate those measurements with the image in a database.

12. Focus position indicator, which annotates each image in a sequencewith the focus position at which it was imaged.

13. Image montage generator, which determines the best-focused image ofeach feature, and generates a composite image or images in which allfeatures are present and each is in focus.

14. Matching engine which correlates a montaged image of an unknownballistic item against images in a database and rank orders orthresholds the results to produce a list of candidate matching items.

15. Examiner workstation 100 which interacts with the ballisticsexaminer and displays the candidate matching item images, feature maps,correlation values, and characteristics of the ballistic items and theircomparisons. The workstation 100 allows him to input queries and receiveresponses in display or hard copy format. It generates, maintains, andaccesses databases of ballistic item images and features and informationannotated by the user. It also includes a highlighter function whichallows examiner to designate features of interest in an unknownballistic item, for matching against a database. The highlighter canutilize a light pen, touch screen display, graphics tablet, or otherinteractive device.

16. For Visible image matching, the apparatus also includes a videomicroscope with matching lenses to the IR microscope, and a database ofvisible images.

17. Image processor subsystem acquires images from the cameras includingeither direct digital output or digitized analog output of the infraredand visible cameras. It standardizes image orientation includingrotation, translation, and alignment based upon features in the image.

BRIEF DESCRIPTION OF THE DRAWINGS

Many details and advantages of the present invention will be apparent tothose skilled in the art when this document is read in conjunction withthe attached drawings where matching reference numbers are applied tomatching items and where:

FIGS. 1 a, 1 b, and 1 c are visible images of the primer areas ofsibling casings. The image illustrate the effects of illuminationvariations and artifacts. In particular, the firing pin indentations inthe centers lack any details, and each shows glint from theillumination. Each image is oriented based upon the breech face marksand the position of the firing pin indentation. 1 a has the bestdetailed primer area. The illumination of 1 b causes much of the breechface markings to be lost, and reverses the appearance of the feature at10:00 from a white to a dark line inside a grey area. The firing pinindent also appears smaller than in a. In c, a slight variation in theillumination angle make the firing pin indentation appear to be raisedup instead. Turning the image upside down makes it appear to be anindentation; however then the position of the indentation is incorrect.Depending on the match engine, these siblings may not be detected asmatches based upon these visible images due to the illumination-inducedvariations.

FIG. 2 shows the apparatus for a dual band IR/visible imaging andmatching ballistic identification system. The split screen shows two IRimages being compared. A four-way split screen can display thecorresponding visible images also at the same time.

FIG. 3 is an infrared image of a shell casing end clearly showing themanufacturers marks and weapons-related marks. This item was heated to105° F.

FIG. 4 is an infrared image of a shell casing end at ambienttemperature, with focus set to best detail the firing pin indentation.

FIG. 5 is an infrared image of the edge of a shell casing at ambienttemperature, with focus set for an ejector mark.

FIG. 6 is an infrared image of a shell casing at ambient temperature,with focus set for a distinctive tool mark.

FIG. 7 is an infrared image of a shell casing at ambient temperature,with focus set for a distinctive tool mark in the primer area.

FIG. 8 is a magnified infrared image of the firing pin indentation in ashell casing at room temperature.

FIG. 9 a and 9 b are magnified infrared images of firing pinindentations of siblings.

FIG. 10 a, b, c, and d show the resulting infrared images of shellcasings at differing temperatures. Certain features are more apparent atdifferent temperatures.

FIG. 11 a,b,c illustrates the removal of the manufacturers markings fromthe casing image prior to matching.

FIG. 12 illustrates minutiae obtained from FIG. 11. In this example,minutiae were defined as pixels whose nearest neighbors had grey valuesdiffering by more than 8 gray levels. Matching can be performed basedupon the minutiae, analogous to matching of fingerprint minutiae.

FIG. 13 presents a block diagram of the apparatus configured with thefollowing subsystems:

(1) image capture and display, including the ballistic examiner'sdisplay, feature highlighter, and interactive devices. (2) featureextraction and characterization. (3) database matching and updating.

FIG. 14 presents a hypothetical graph of the apparent size of a featureas a function of gray scale values (or temperatures) included in thefeature, using different spectral bands. Such graphs can be used toautomatically determine the material of the item including principal andtrace components, the type of weapon, and the manufacturer.

DETAILED DESCRIPTION OF THE FIRST EMBODIMENT—USING INFRARED IMAGES TOREFINE VISIBLE IMAGE DETAILS FOR BALLISTICS IDENTIFICATION IMAGE ANDINFORMATION CAPTURE

A ballistic item 10 is placed in a holder 12 and surrounded by a shroud14 covering it and an infrared camera 16 such that external lighting andheating sources are blocked. The camera focus mechanism 18 is variedover a range such that all details of the infrared image of theballistic item are clearly captured in a sequence of infrared images 20.

Optionally, one or more spectral filters 30 can be installed andadditional sequences of infrared images obtained. Optionally, a heater32 can be used to raise the temperature of the ballistic item andadditional sequences of infrared images obtained for the item at theelevated temperature and as it cools.

A controlled light source 110 is then turned on to illuminate theballistic item and a video camera 116 is used to produce a sequence ofvisible images 120 by varying the focus mechanism 118.

Each image is annotated with date and time, workstation #, itemtemperature, focus setting, and item reference number. The focus andimage capture processes can be automated such that a succession ofminute variations in focus is performed and an image taken at each step,or the focus and image capture can be manually controlled using anexaminers workstation 100 consisting of a display screen 22 and inputcontrols 24 including any combination of keyboard, mouse, voice, orsimilar device. The workstation 100 also contains highlighting device 90for manually specifying areas of images or textual information ofparticular interest to the examiner. The highlighter can be anycombination of touch screen, lightpen, graphics tablet, or similardevice. The display has the ability to mosaic several infrared 20 andvisible 120 images on a single screen.

Text information is entered which identifies the ballistics item andrelated information such as case #, weapon type, ammunition type,location where found, etc. That information can be read from an evidencetag using a bar code reader 28 or input through the controls of theexaminers workstation 100 such as by keyboard. The text information canbe displayed on the screen 22 along with the corresponding annotatedimage.

Both text information and image sequences are stored in an infraredimage sequence database 26 and a visible image sequence database 126.

Feature Extraction and Characterization

The sequences of visible images are processed to extract andcharacterize apparent features at 142 using any of various standardautomated image processing techniques or by manual highlighting by theexaminer. Characterization a minimum includes the relative positions offeatures, their shape, their area and perimeter length, and variation ingray scale distribution within the feature. Similar processing isperformed at 42 on the infrared sequences using the same or similarprocessing.

For each visible feature, one image is selected from the sequence inwhich that feature is in sharp focus. The feature area of that image isextracted and used to create a montaged visible image 138 in which eachfeature is shown and each is in sharp focus. The same is done for theinfrared sequence to create a montaged infrared image 38. Each montagedimage is associated with the characterizations of the associatedfeatures at 42 and 142.

Image Comparison and Corrections to Visible Images

The characteristics of the montaged visible image 138 and montagedinfrared image 38 are compared at 136. Any feature of the visiblemontaged image 138 which does not have a corresponding infrared montagedfeature in image 38 is removed from image 138. Correspondence requiresat a minimum overlapping locations and similar shapes. Those removedvisible features are considered to be illumination-induced artifacts.Any visible feature which has a corresponding infrared feature, butwhich differs in gray scale distribution from the IR feature issubjected to variation in its gray scale allocation or LUT 134 to findthe best correlation with the corresponding infrared feature. Thosevisible features are considered to have illumination-induced gray scalereversals. Any infrared feature which does not have a correspondingvisible feature is considered to represent a possible true featurehidden in the visible image due to shadow. At the option of theexaminer, such features can be (1) copied from the infrared montagedimage to the visible montaged image, (2) annotated such as in color onthe visible montaged image, (3) tested by the use of spectral filtersand/or heating to determine if it results from residue, in which casethe item should be cleaned and re-imaged.

The resulting montaged infrared and visible images, along with theircharacteristics and textual information are entered into a database ofinfrared characterizations 50 and enhanced visible characterizations 150of unknown ballistic items. The enhanced visible characterization canthen be used with current ballistic identification methods andapparatus, producing more accurate results due to the elimination ofillumination-induced artifacts, and the detection of hidden features dueto shadow.

DETAILED DESCRIPTION OF THE SECOND EMBODIMENT—BALLISTICS IDENTIFICATIONTHROUGH MATCHING INFRARED CHARACTERISTICS

When infrared imaging alone is used for ballistics identification,without use of corresponding visible images, then components 116, 118,120, 126, 138, 142, 134, 150 and references to visible images,sequences, features, and montages can be eliminated from the apparatusdescription.

Image and Information Capture

A ballistic item 10 is placed in a holder 12 and surrounded by a shroud14 covering it and an infrared camera 16 such that external lighting andheating sources are blocked. The camera focus mechanism 18 is variedover a range such that all details of the infrared image of theballistic item are clearly captured in a sequence of infrared images 20.

Optionally, one or more spectral filters 30 can be installed andadditional sequences of infrared images obtained. Optionally, a heater32 can be used to raise the temperature of the ballistic item andadditional sequences of infrared images obtained for the item at theelevated temperature and as it cools.

Each image is annotated with date and time, workstation #, itemtemperature, focus setting, and item reference number. The focus andimage capture processes can be automated such that a succession ofminute variations in focus is performed and an image taken at each step,or the focus and image capture can be manually controlled using anexaminers workstation 100 consisting of a display screen 22 and inputcontrols 24 including any combination of keyboard, mouse, voice, orsimilar device. The workstation 100 also contains highlighting device 90for manually specifying areas of images or textual information ofparticular interest to the examiner. The highlighter can be anycombination of touch screen, lightpen, graphics tablet, or similardevice. The display 22 has the ability to mosaic several images on asingle screen.

Text information is entered which identifies the ballistics item andrelated information such as case #, weapon type, ammunition type,location where found, etc. That information can be read from an evidencetag using a bar code reader 28 or input through the controls of theexaminers workstation 100 such as by keyboard. The text information canbe displayed on the screen 22 along with the corresponding annotatedimage.

Both text information and image sequences are stored in an imagesequence database 26.

Feature Extraction and Characterization

The sequences of infrared images are processed to extract andcharacterize apparent features at 42 using any of various standardautomated image processing techniques or by manual highlighting by theexaminer. Characterization at a minimum includes the relative positionsof features, their shape, their area and perimeter length, and variationin gray scale distribution within the feature.

For each apparent infrared feature, one image is selected from thesequence in which that feature is in sharp focus. The feature area ofthat image is extracted and used to create a montaged infrared image 38in which each feature is shown and each is in sharp focus. If there isthe chance that debris may remain on the ballistic item, then apparentfeatures may in fact be debris. This event can be tested for bycomparing corresponding focus images using different spectral filters30. Apparent features which persist through a range of filters areconsidered actual features. Each montaged image is then associated withthe characterizations of the associated features at 42.

Identification using Matching

When the unknown ballistic item characterized in 50 has been identifiedor linked with other known items, by a ballistics examiner or anautomated system, its file is move to relational database 60 which thenincludes (1) textual information related to the case, its disposition,location of the item, names and dates, weapon information and links toother forensic and law enforcement databases; (2) images which mayinclude the montaged feature images alone or also the image sequences,and may also include highlights created by a ballistics examiner andcorresponding notes, which should be displayed in color for emphasis;and (3) feature characteristics including a summary of the number andtypes of features seen, and details on each feature including type,shape, size, location, and variation in gray scale and edge effects.

Database 60 will contain characterization of known or linked ballisticitems. When an unknown item 10 is presented for identification, it isprocessed as detailed above to produce its characterization at 50. Theresulting characteristics are used to select initial candidates from thedatabase 60 based upon text, image, and feature characteristics whichare relatively immune to error or variation. For example, the calibre ofammunition. In matching shell casings, if the unknown firing pinindentation is centered, only database entries with centered firing pinsare considered as potential candidates. The presence or breech facemarkings, ejector or extractor marks may also be considered relativelyimmune to error or variation.

The initial candidate matches are then further processed using the textmatching engine 70 which might provide for example the date ofmanufacture of the weapon, meaning that all ballistic items collectedprior to that date need not be considered as matches. Other informationis compared and scored as to similarity, such as information about thetype of crime associated with the ballistics item, the locale where theitem was collected, the presence of other similar items at the samecollection, etc. The similarity score will generally not excludecandidates from further consideration, but may influence their rankordering in presentation to a ballistics examiner for considerationbelow.

Remaining candidate matches are then further processed using the featurematching engine 72 which first compares summary characteristics such asthe number and type of features, and then compares the details of eachfeature of the unknown and candidate items. Various metrics or scoringtechniques can be defined to calculate a goodness of match based uponthe feature characteristics. Fingerprint matching and face matchingtechnique are especially suitable, with Prokoski et al. U.S. Pat. No.5,163,094 as an example. Candidates which do not correlate sufficientlywith the unknown item are not longer considered.

Remaining candidate matches are then further processed by correlatingthe montaged images using an image processing technique such as ProkoskiU.S. Pat. No. 6,173,068 to produce an image correlation value. Otherimage processing comparison techniques may be used. The remainingcandidates are then rank ordered based upon their text similarity score,their feature characteristics goodness of match, and their imagecorrelation value. Various algorithms for performing the rank orderingcan be used. An effective one is to assign a rank ordering based on eachof the three measures, and then perform a composite rank ordering basedupon the ordinal sum of the component rank orderings.

For various reasons it may be desirable to use only one or two of thethree comparison techniques.

The top placing candidates are presented to the ballistics examiner forreview on the display 22. Using interactive controls 24 he canmanipulate the imagery to perform further visual matching ofsimultaneously displayed unknown and candidate text, featurecharacteristics, and imagery. His decision 78 for each candidate iseither that it is a match or is not a match for the unknown item, andthe information in database 50 is transferred to database 60. When hedetermines that a match has occurred, then he indicates through thecontrol mechanism 24 that the previously unknown item is now linked to aparticular candidate 80, If he exhausts the top placing candidates, hethen indicates that the unknown item has no match 82 in the existingdatabase as of the current date.

1. Method for characterizing a ballistic item including the steps: a.Produce a sequence of digitized passive infrared images of the item atdifferent focus points such that the deepest to the highest features areeach in sharp focus in at least one image; b. tag each image with the IDof the item; c. tag each image with specifics of the imaging set-upincluding the focus position, and size of digitized image array; d.store the tagged image sequence in a database.
 2. Method of claim 1 withthe additional step: e. produce a photomontage from the tagged imagesequence in which each section of the montage image is the correspondingsection of the image from the tagged sequence in which that section isin sharpest focus.
 3. Method of claim 1 with the additional step ofreplacing each tagged image in the sequence with a tagged extractedfeature image containing only features at least a specified sizeextracted from the tagged image.
 4. Method of claim 1, furthercomprising steps: adding to each tag weapon-specific ancillaryinformation including calibre, type of ammunition, direction of twist,number of lands, serial number; and adding to each tag incident-specificinformation including type of crime committed, location where item wasfound, associated names, method of crime.
 5. Method for identificationof a ballistic item including the steps: a. characterize the unknownballistic item by producing a tagged image sequence b. compare the imagesequence with those contained in a database c. determine those sequencesin which one or more images are similar to the unknown tagged imagesequence d. display the similar pair(s) of images to a ballisticsexaminer who reviews the display and rules that the unknown ballisticitem is a match to an item in the database if the similar pair(s) ofimages are sufficiently alike.
 6. Method of claim 5, wherein step d isperformed automatically by further image processing.
 7. Method of claim5 step, further comprising the steps: comparing the weapon-specific andincident-specific tagging information of the similar image pair(s);displaying the similarities and dissimilarities in the tagginginformation along with the images for further consideration by aballistics examiner who reviews the display and rules that the unknownballistic item is a match to an item in the database if the tagginginformation as well as the similar pair(s) of images are sufficientlyalike.
 8. Method of claim 7 wherein the reviewing by the ballisticsexaminer is performed automatically by further image processing. 9.Method for identification of a ballistic item including the steps: a.characterize the unknown ballistic item by producing from passiveinfrared images a tagged extracted feature sequence; b. compare theextracted feature sequence with those contained in a database; c.determine those sequences in which one or more extracted features aresimilar to the unknown extracted feature sequence; d. display thesimilar pairs of extracted features to a ballistics examiner who reviewsthe display and rules that the unknown ballistic item is a match to anitem in the database if the similar pair(s) of extracted features aresufficiently alike.
 10. Method of claim 9, wherein the reviewing by theballistics examiner is performed automatically by further imageprocessing.
 11. Method of claim 9, further comprising the steps:comparing the weapon-specific and incident-specific tagging informationof the similar extracted feature pair(s): and comparing the similaritiesand dissimilarities in the tagging information along with the extractedfeatures for further consideration by a ballistics examiner who reviewsthe display and rules that the unknown ballistic item is a match to anitem in the database if the tagging information as well as the similarpair(s) of extracted features are sufficiently alike.
 12. Method ofclaim 11, wherein said comparing step is performed automatically byfurther image processing.
 13. Method of claim 1, further comprising thesteps: heating or cooling the ballistic item to vary its temperature;producing an image sequence in which both focus and temperature arevaried; and tagging each image with the corresponding temperature.14-17. (canceled)
 18. Method to detect residue on a ballistic item,including the steps: a. apply a sequence of spectral filters to the IRcamera; b. for each filter, vary the focus to produce an image sequenceof passive infrared images; c. extract features from each image in thesequence; d. compare the feature sets in images which have the samefocus setting but different spectral filters; e. display those featureswhich are filter-sensitive as possible residue; f. Annotate featureswith likely type of residue based upon the filter sensitivity. 19.Apparatus for characterizing a ballistic item including: a. IR camerawith lenses and focus control; b. Mechanism for varying the focuscontrol to produce a sequence of images; c. Image digitizer and storage;d. Mechanism for tagging images with ancillary information; e. Featureextractor; f. Processor for characterizing the features; g. Processorfor creating a montage; h. Display; i. Mechanism for positioning theitem within the camera field of view.
 20. The apparatus of claim 19including also the elements: j. device for heating the ballistic item; kdevice for measuring the temperature of the ballistic device; l. devicefor applying optical filters before the camera lens; m. mechanism fortagging the resulting images with temperature and filter data. 21.Apparatus for identifying a ballistic item including the elements of 20plus: n. Database of characterized ballistic items; o. Database matchingengine; p. Output or display mechanism.
 22. Method for identifying aballistic item including the steps a. for each montaged infrared imagederived from an unknown item database and a known item database producea relative location map as follows: b. compute the centroid location foreach feature, including all striations, gouges, breech face marks, andfiring pin indents, where each striation is considered a separatefeature; c. Compute the distances between each pair of centroids; d. Tagthe distances with the type of feature represented at each end; e. Matchthe list of tagged distances with corresponding lists for the known itemdatabase.
 23. Method for separating manufacturers marks from weaponsmarks in ballistic images, including the steps: a. heating the ballisticitem to enhance the manufacturers marks; b. producing a passive infraredimage of the ballistic item; c. producing a template of the areascontaining the enhanced manufacturers marks; and d. extracting thetemplate areas to form an image containing the manufacturers marks,wherein the remaining image contains no manufacturers marks but containsweapons marks which did not overlay manufacturers marks.
 24. Method forestimating the volume of groves, indentations and striations inballistic items including the steps: a. heat the ballistic item to anelevated temperature; b. measure the mean temperature of the item usinga radiometric infrared camera; c. capture a sequence of passive infraredimages as the item cools; d. capture for each image the mean temperatureusing the radiometric camera; e. estimate the volume of each depressedfeature by determining its heat loss rate relative to the meantemperature of the item, considering also the materials composition ofthe item.